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Performa nee- Based 
Scholarships 

EMERGING FINDINGS FROM A NATIONAL DEMONSTRATION 

By Reshma Patel and Lashawn Richburg-Hayes 



I ncreasing the academic success of 
students in college is a national 
imperative. While college enrollment has 
increased from 5.9 million students in 
1965' to 20.4 million in 2009, 2 
graduation rates remain low. Just over 
half of students seeking a bachelor’s 
degree and enrolled in a four-year 
institution full time in fall 2002 had 
completed their degree within six years. 3 
Despite the marked increase in access to 
college since the passage of the Higher 
Education Act of 1 965 — which extended 
need-based financial assistance to the 
general population for the first time — 
more work remains to be done to 
improve persistence in college and 
graduation rates. 

Low-income students are at particular 
risk of not persisting to earn a 
certificate or degree, often because of 
competing priorities, financial 
pressures, and inadequate preparation 
for college. Financial aid may improve 
access to and persistence in college for 
this population. Research suggests 
that financial aid is positively 
associated with increased enrollment 4 
and with increased persistence. 5 While 
most of the evidence is not causal, one 
random assignment study that 
provided need-based grants to 
students attending public universities 
in Wisconsin found modest impacts on 
some academic measures. 6 



One form of financial assistance 
designed explicitly to reward academic 
success among low-income students is a 
performance-based scholarship, paid 
contingent on attaining academic 
benchmarks. Unlike merit-based aid, 
performance-based scholarships focus 
on current and 
future performance 
rather than earlier 
accomplishments. 

MDRC’s Opening 
Doors study of 
performance-based 
scholarships in 
Louisiana showed 
that such a program 
had positive effects 
on students’ credit 
accumulation, 
grades, and 
persistence in 
college. 7 The program targeted low- 
income parents, and the study sample 
was comprised of relatively older, 
unmarried, and mostly female students. 
Unfortunately, the devastation inflicted by 
Hurricane Katrina occurred during the 
study, making it difficult to confirm the 
program’s long-term effects. Building on 
these promising findings, MDRC 
launched the Performance-Based 
Scholarship (PBS) Demonstration in four 
states in 2008, 8 followed by an additional 
two states in 2010. This brief 
summarizes the early findings of the 
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study, which show modest but positive 
effects on important markers of academic 
progress. 

HOW ARE PERFORMANCE- 
BASED SCHOLARSHIPS 
SUPPOSED TO WORK? 

By conditioning additional financial aid on 
certain performance benchmarks, the 
programs seek to encourage students to 
focus more on their studies, which, in turn, 
should lead them to perform better in their 
classes in the short term. In the medium 
term, they should progress through their 
degree requirements more quickly by 
increasing their term-to-term enrollment 
and their credits attempted and earned. 
Increases in these academic outcomes may 
then lead to long-term gains, including 
year-to-year persistence, more total 
cumulative credits earned, and graduation 
or transfer to a four-year or more selective 
college. Finally, if the effects on academic 
outcomes remain positive and strong, the 
intervention could lead to improved labor 
market outcomes, including higher 
earnings. 

To this end, the project seeks to answer 
several questions: 

• What is the impact of performance-based 
scholarships (in various settings with 
different types of students) on short-term 
academic outcomes, including credit 
attempts, credit completion, and grade 
point averages (CPA)? 

• What is the impact of the scholarships 
on longer-term academic outcomes, 
including rates of persistence in school 
and attainment of degrees and 
credentials? 



• How does variation in the amount and 
duration of the scholarships affect 
academic outcomes? For example, how 
do the impacts of a $i,ooo scholarship 
over one term compare with those of a 
$i,ooo scholarship over one year? 

• For which types of students do the 
scholarships work best? 

• How do the scholarships affect students’ 
academic motivation and behavior? What 
effect do they have on how students 
spend their time and on their mental 
well-being? 

THE DESIGN OF THE 
PERFORMANCE-BASED 
SCHOLARSHIPS IN 
EACH STAT E 

Table l summarizes the eligibility criteria 
and design variations for the programs in 
the six states in the PBS Demonstration. 
While the programs in each state vary by 
target population, performance 
benchmarks, scholarship amounts, and the 
integration of student services (among 
other things), there are some 
commonalities. All of the programs target 
low-income students, and the scholarships 
are paid directly to students, allowing them 
to use the funds for their most pressing 
needs, whether books, child care, or other 
financial obligations that may disrupt their 
studies. Importantly, the scholarships are 
paid in addition to Pell Grants — the main 
federal source of need-based aid — and 
other existing financial aid programs. In 
this way, students have more funds to cover 
academic and living expenses and can 
potentially reduce their dependency on 
loans . 9 Lastly, unlike merit-based aid, 
performance-based scholarships are paid to 
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Table 1: Design of the Performance-Based Scholarships in Each State 





OHIO 


NEW MEXICO 


NEW YORK 


CALIFORNIA 


ARIZONA 


FLORIDA 


ELIGIBLE 

POPULATION 


• Age 18+ 

• Parent 

. Zero EFC 1 


• Age 17 to 20 

• Freshmen 

• Pell-eligible 


• Age 22 to 35 

• Live away from 
parents 

• Require 
developmental 
education 

• Pell-eligible 


• Age 16 to 19 

• High school 
seniors applying 
for financial aid 

• Below Cal Grant 
A/C income 
threshold 2 


• Hispanic male 

• Less than 45 credits 
earned 

• EFC below $5,273 3 


• Age 18+ 

• In need of 
remedial math 

• EFC below 
$5,273 


MAXIMUM 
SCHOLARSHIP 
AMOUNT PER 
TERM 


$600 (quarter 
institutions) to 
$900 (semester 
institutions) 


$1,000 


$1,300 


$333 (quarter 
institutions) or 
$500 (semester 
institutions) to 
$1,000 


$1,500 


$600 


SCHOLARSHIP 

DURATION 


2 semesters or 3 
quarters 


4 semesters 


2 full semesters 
and l summer 
semester 4 


l term to 2 years 


3 semesters 


3 semesters 


MAXIMUM 

TOTAL 

SCHOLARSHIP 

AMOUNT 


$i,8oo 


$4,000 


$2,600 to $3,900 


$1,000 to $4,000 


$4,500 


$1,800 


ACADEMIC 

BENCHMARKS 


Part time: 6 to 11 
credits with “C” or 
better in each 
Full time: 12 or more 
credits with “C” or 
better in each 


Complete 12 or 
more credits (1 st 
semester) or 15 
credits (subsequent 
semesters) with a “C” 
or better average 


6 or more credits 
with “C” or better 
in each 


Complete 6 or more 
credits with a “C” 
or better average 


Part time: 6 to n 
credits with “C" or 
better in each 
Full time: 12 or more 
credits with “C” or 
better in each 


Complete a 
sequence of 
math courses 
with a "C” or 
better in each 
course 


ADDITIONAL 

SERVICE 

CRITERIA 


None 


Meet with adviser 


None 


None 


Meet with adviser, 
complete tutoring 
and workshop 
requirements 


Complete 

tutoring 

requirements 



1 The EFC (Expected Family Contribution) is the amount of money a family is expected to be able to contribute to a student’s education, 
as calculated according to federal guidelines. Students with a zero EFC are eligible for federal Pell Grants. 

2 Cal Grant is a financial aid program funded by the state of California. The awards do not have to be paid back, but to qualify students must fall below 

the income and asset ceilings. Cal Grants A and C have the same income and asset ceilings. MDRC used this income ceiling as a cutoff for eligibility in California. 

3 Students with an EFC of up to $5,273 during the 2010-2011 year are eligible for federal Pell Grants. 

4 Half of the program group in New York are eligible for an additional summer performance-based scholarship. 
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SOURCE: MDRC calculations from Delgado Community College, Lousiana Technical College-West Jefferson, Ohio Board of Regents, University of New 
Mexico, Borough of Manhattan Community College, and Hostos Community College transcript data. 

NOTES: Rounding may cause slight discrepancies in sums and differences. 

A two-tailed t-test was applied to differences between research groups. 

Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Estimates are adjusted by cohort and campus. 
a Louisiana results are from MDRC's Opening Doors Demonstration. 
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students based on their academic 
performance in the current term, regardless 
of what happened in previous terms. 

Each state tests the scholarships in a 
somewhat different way or with a different 
target population, as shown in Table 1. For 
example, the study in Ohio focuses on 
low-income parents and offers a part-time 
and full-time scholarship award at one of 
three community colleges. Conversely, in 
New Mexico, the sample consists of a 
traditional, college-going population at a 
four-year institution, with more stringent 
academic benchmarks to give students an 
incentive to graduate on time. In New York 
and Florida, the study targets students in 
need of developmental, or remedial, 
education; the study in Florida is designed 
to give students an incentive to pass a 
sequence of developmental math courses. 

A range of additional student services, such 
as tutoring and advising, are also linked to 
the scholarships in Arizona, Florida, and 
New Mexico, and the program in Arizona is 
a particularly cohesive program targeted 
toward a Hispanic male population. Lastly, 
the study in California is testing a statewide 
program, with a portable scholarship of 
varying amount and duration that students 
can take to any degree-granting, accredited 
postsecondary institution in the country. 



(for instance, rates of persistence) can be 
attributed with a high level of confidence to 
the PBS program rather than to the types 
of students who enroll in the program. 

In addition to the states listed in the table, 
MDRC is also working with the UNCF 
(United Negro College Fund), the nation’s 
largest minority education organization, to 
provide technical assistance in 
implementing performance-based 
scholarships at three UNCF member 
institutions: Benedict College and Claflin 
University in South Carolina and Florida 
Memorial University. 



EARLY FINDINGS FROM OHIO, 

NEW YORK, AND NEW MEXICO 
MDRC now has preliminary results from 
performance-based scholarship programs 
in Ohio, New York, and New Mexico, in 



addition to the original 
results in Louisiana. 10 While 
the effects vary across these 
sites, there are some 
commonalities in the 
emerging findings. All of the 
impacts described below are 
statistically significant, 
indicating that the differences 
reported are likely to be 
caused by the programs 
rather than by chance. 11 



All the oiteo where 
performance -bowed 
ocholarolupo were 
tea ted found impacts 
on credit d earned in one 
or more oemeotero or 
ter mo. 



Each study employs a random assignment 
research design, the “gold standard” in 
program evaluation. Random assignment 
creates two groups of students that are 
similar in characteristics that can be 
measured, such as age and gender, and in 
those that are more difficult to measure, 
such as tenacity and motivation. As a 
result, subsequent differences in outcomes 



• Increases in credits earned (see Figure 1). 

All the sites found impacts on credits 
earned in one or more semesters or 
terms. The Louisiana study saw an 
increase in credits earned, averaging 3.5 
credits over four terms (not shown in 
figure). In the Ohio program, which has a 
target population similar to that of 
Louisiana, the program group students 
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SOURCE: MDRC calculations from Delgado Community College, Lousiana Technical College-West Jefferson, Ohio Board of Regents, 

University of New Mexico, Borough of Manhattan Community College, and Hostos Community College transcript data. 

NOTES: Rounding may cause slight discrepancies in sums and differences. A two-tailed t-test was applied 
to differences between research groups. 

Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Estimates are adjusted by cohort and campus. 

Measures chosen are proxies for the benchmark required to earn the full performance-based scholarship, specific to the program in each state. 

In Louisiana, the end-of-term benchmark was earning 6 or more credits with an overall GPA of 2.0 or higher, and the measure shown above 
represents the proportion of students earning a GPA of 2.0 or higher. In Ohio, the full-time benchmark was earning 12 or more credits with a “C" 
or better in each, and the measure shown above represents the proportion of students earning 12 or more credits. In New York, the end-of-term 
benchmark was earning 6 or more credits with a “C” or better in each, and the measure shown above represents this exactly. In New Mexico, the 
end-of-term benchmark was earning 12 or more credits (first term) or 15 or more credits (second term) with an overall GPA of 2.0 or higher, and 
the measure shown above represents this exactly. 
a Louisiana results are from MDRC's Opening Doors Demonstration. 
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Mexico, but the scholarship requires that 
students take 15 credits in their second 
term — which is what they consistently 
need to complete in order to graduate on 
time in four years. As a 
result, the program in New 
Mexico had a 25.6 
percentage point impact on 
second-term enrollment. 

• Increase in students’ 
abilities to meet end-of- 
term benchmarks during 
program terms (see Figure 

2). In Louisiana, students 
had to earn at least six credits with a CPA 
of 2.0 or higher in order to get the end- 
of-term award. There was a 12.1 
percentage point increase in the 
proportion of students earning a GPA of 
2.0 or higher in the first semester of the 
program, and a 20.8 percentage point 
increase in the second semester of the 
program. 



In all of the oiteo, 
ao a reoult of the 
ocholarobipo there 
were increased In tbe 
oecond term in credits 
attempted and/or fall- 
time enrollment. 



earned an average of about two full 
credits more than the control group 
students over two terms of study. The 
New York site, which targets independent 
students in need of developmental 
education, had an increase of slightly 
more than half a credit earned in the first 
term (results from later terms will be 
available in a forthcoming report). The 
New Mexico study, which is the only one 
housed at a four-year institution, found 
no impact on credits earned over the first 
term, but showed an increase of 0.6 
credits in the second term. While these 
impacts seem fairly modest, the increase 
in credits can sometimes account for one 
full course toward a student’s degree 
requirements, essentially shortening the 
time to degree completion by that 
amount. 

• Greater impacts in the second term. All 

the sites, including the original Louisiana 
study, showed an increase in credits 
attempted and/or full-time enrollment in 
the second term. In Louisiana, there was 
an increase of 1.2 credits attempted in 
the second term, and a 15.3 percentage 
point increase in full-time enrollment. 
Similarly, in Ohio, program participants 
showed an increase of 0.6 credits 
attempted in the second term, and a 6.3 
percentage point increase in full-time 
enrollment. In New York, while there was 
no increase on credits attempted in the 
second term, the program did have a 7.4 
percentage point increase in full-time 
enrollment. Lastly, in New Mexico, 
students in the program attempted 
almost one full credit more than control 
group members in the second term. 
There was no increase in full-time 
enrollment (12 credits per term) in New 



Students in Ohio had to earn at least 12 
or more credits with a “C” or better in 
each for the full-time award. While 
detailed grade data are not available, 
using a proxy measure of earning 12 or 
more credits, there was a 7 percentage 
point increase in the first term of the 
program and a 12.7 percentage point 
increase in the second term of the 
program. 

In New Mexico, students had to earn 12 or 
more credits (in the first term) and 15 or 
more credits (in subsequent terms), with 
a C PA of 2.0 or higher. While there was no 
impact on meeting the benchmark for the 
first term, there was a 16.4 percentage 
point increase in the proportion of 
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students earning the end-of-term 
benchmark for the second term of the 
program (more than a 50 percent 
increase). These impacts are striking, as 
the outcome is the academic benchmark, 
specific to the program in each state, that 
the scholarship was meant to influence. 
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• Varied effects on term-to-term 
persistence. While the Louisiana program 
saw sizeable impacts on rates of 
registration in virtually every term after 
random assignment, the early findings in 
Ohio, New York, and New Mexico have 
not found similar effects. In part, this is 
owing to high rates of persistence for 
control group students in the newer 
sites, providing a bar that is difficult to 
surpass. In addition, while 
the Louisiana program took 
place during an economic 
boom (2004 to 2005), the 
PBS programs began during a 
period of economic downturn 
(2008 to 2010). Ohio, New 
York, and New Mexico also 
have more generous financial 
aid options for the average 
low-income student than did 
Louisiana at the time of the 
Opening Doors study, 
meaning that control group students in 
those states were more likely to have 
other forms of aid. Additionally, it may be 
that the target group in Louisiana was 
particularly well-suited to the 
intervention. Even though the study has 
not yet found effects on persistence in 
the PBS Demonstration sites, there 
continue to be impacts on other 
academic outcomes. 



• Debt reduction. The studies of both the 
Ohio and New Mexico programs found 



evidence that students’ debt was reduced 
as a result of the scholarships. Loans 
made up a smaller proportion of total 
financial aid for program group students 
in Ohio and New Mexico than for control 
group students. 

These mostly short-term results suggest that 
performance-based scholarships can move 
the dial on some important markers of 
academic success. If the programs can show 
lasting effects after the scholarships are no 
longer available — and impacts on 
persistence emerge in later terms — these 
scholarships could lead to higher graduation 
rates and translate into higher earnings. 
MDRC will follow these longer-range 
outcomes closely in Ohio, New York, and 
New Mexico in the coming years. In 
addition, forthcoming results from three 
more states in the PBS Demonstration — 
California, Arizona, and Florida — will add to 
the body of knowledge on the effectiveness 
of these scholarships on improving 
academic success for low-income students. 

POTENTIAL IMPLICATIONS 

Performance-based scholarships are paid 
in addition to Pell Grants and other state 
and local financial aid for students, and 
thus their results can speak only to the 
impact of their use on top of the existing 
aid structure. However, at a time when 
policymakers are looking for ways to make 
financial aid more effective, there are some 
notable differences in the delivery of 
performance-based scholarships that could 
be relevant to current and future aid and 
scholarship programs: 

• Performance-based scholarships are paid 
directly to students, potentially making 
them more meaningful to students than 
Pell Grants, for example, which are often 
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paid directly to the institution that 
students are attending. 

• Performance-based scholarships have an 
added component of frequent 
communication with students, even in the 
versions of the demonstration that do not 
have an advising or student services 
component. A coordinator at each site is 
responsible for following up with students 
and serving as a central point of contact in 
case of questions. In this way, an aspect of 
student support is built into financial aid. 

• Performance-based scholarships are 
generally paid in increments over the 
semester, when possible. This means 
that students get their aid over the entire 
semester, rather than in a large lump 
sum, as most grants and scholarships 
award aid. There are potential benefits to 
this method, as students may be 
encouraged to consistently work toward 
an end goal while receiving modest 
benefits along the way. In addition, 
students may be able to make better 
financial decisions throughout the term 
with this type of disbursement schedule. 

Importantly, the modifications discussed 
above can be made with very little added 
cost, if any. Additionally, states and local 
governments may want to consider 
performance-based scholarships as a 
supplement to the current financial aid 
offered to low-income students. 

WHAT’S NEXT? 

The research team will be releasing 
periodic findings until the project ends in 
December 2014. These findings will include 
a full implementation study, a qualitative 
study, a more detailed impact study, and a 
cost study. 



FUNDERS AND PARTNERS OF THE PERFORMANCE- 
BASED SCHOLARSHIP DEMONSTRATION 



The launch of the Performance-Based Scholarship 
Demonstration was made possible by the generous support 
of the Bill & Melinda Gates Foundation. The operation and 
research for the demonstration at various sites was enabled by 
support from: 

• The Bill and Melinda Gates Foundation 

• The College Access Foundation of California 

• Open Society Foundations 

• The Helios Education Foundation 

• The Joyce Foundation 

• The Kresge Foundation 

• New York City Center for Economic Opportunity 

• The Ohio Department of Job and Family Services through 
the Ohio Board of Regents 

• The Robin Hood Foundation 

• Institute of Education Sciences, 

U.S. Department of Education 

• California Student Aid Commission 

• The City University of New York: Borough of Manhattan 
Community College and Hostos Community College 

• Hillsborough Community College 

• Lorain County Community College 

• The Los Angeles Chamber of Commerce 

• Owens Community College 

• Pima Community College 

• Sinclair Community College 

• UNCF 

• University of New Mexico 
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NOTES 

1 Snyder, Dillow, and Hoffman (2008). 

2 U.S. Department of Education (2011a). 

3 U.S. Department of Education (2011b). 

4 St. John, Musoba, Simmons, and Chung 
(2002); Kane (2004); Dynarski (2000, 2003); 
Cornwell, Mustard, and Sridhar (2006). 

5 St. John, Kirshstein, and Noell (1991); St. John, 
Hu, and Weber (2001); DesJardins, Ahlburg, and 
McCall (2002); Bettinger (2004); Singell and 
Stater (2006). 

6 Goldrick-Rab, Harris, Benson, and Kelchen 
(2011) have experimental evidence of providing 
additional aid without any prior academic criteria 
or academic benchmarks. They find that 
recipients of the need-based grant were 28 
percent more likely than other Pell Grant 
recipients to have earned at least 60 college 
credits over a two-year period. 

7 Richburg-Hayes et al. (2009a); Brock and 
Richburg-Hayes (2006); Barrow, Richburg-Hayes, 
Rouse, and Brock (2009). 

8 See Richburg-Hayes et al. (2009b) for an 
overview of the programs in each state. 

9 Financial aid regulations prohibit certain 
students from receiving financial aid in excess of 
their need. In these instances, federal work-study 
or loans may be displaced for the performance- 
based scholarship. 

10 Richburg-Hayes, Sommo, and Welbeck (2011); 
Miller, Binder, Harris, and Krause (2011); Cha and 
Patel (2010); Richburg-Hayes etal. (2009a). 

11 Using a two-tailed t-test applied to differences 
between research groups, all p-values are less 
than or equal to o.io. 
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Performance-Based Scholarships 

Emerging Findings from a National Demonstration 

By Reshma Patel and Lashawn Richburg-Hayes 




ow-income students are at particular risk of not persisting to earn a certificate or 
degree, often because of competing priorities, financial pressures, and 
inadequate preparation for college. One form of financial assistance designed 
explicitly to reward students’ academic success is a performance-based scholarship, 
paid contingent on attaining academic benchmarks. MDRC’s 2009 Opening Doors 
study of performance-based scholarships in Louisiana showed a positive effect on 
students’ credit accumulation, grades, and persistence in college. In 2008, MDRC 
launched the Performance-Based Scholarship Demonstration, a study in six states that 
is testing whether different configurations of these scholarships in different locations 
and among different types of students would be equally effective in improving their 
outcomes. This brief summarizes findings in Louisiana, Ohio, New York, and New 
Mexico, which show modest but positive effects on important markers of academic 
progress. 





